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1.0 Introduction 


Immediately following the power system collapse of August 14, 2003, NPCC formed 
the NPCC 2003 Blackout Investigative Team (BIT) which was given the following 
charge by Mr. Charles J. Durkin, Jr., the Chair of the Northeast Power Coordinating 
Council: 


• Develop a detailed sequence of events which the NPCC Areas 
experienced on August 14, 2003; 

• Develop a detailed analysis of the events that resulted in the cascading 
collapse of a major portion of the NPCC Region on August 14, 2003, 
and a sequence of the restoration of the Areas and areas for 
investigation; and 

• Based on the lessons to be learned, develop findings, conclusions and 
recommendations for the NPCC Areas of investigation, including the 
restoration effort. 

These charges would focus on answering two critical questions: “What went wrong in 
NPCC?” and, second, “What went right?” Recommended revisions or enhancements to 
practices and criteria would follow from the answers to these questions. Further, the 
efforts of the NPCC 2003 Blackout Investigative Team complemented the input to the 
numerous investigations conducted in parallel by the NERC and the U.S.-Canada Power 
System Outage Task Force. 

To ensure the necessary breadth of expertise and authority for such a task, the BIT was 
composed of the following individuals and NPCC groups: 

Members, NPCC 2003 Blackout Investigative Team: 

Edward A. Schwerdt Northeast Power Coordinating Council 

Chair, NPCC BIT 

Officers of the NPCC Reliability Coordinating Committee During the Course of the 
Investigation: 

Colin Anderson Ontario Power Generation, Inc. 

RCC Vice Chair, November, 2003-December, 2005 
H. Kenneth Haase New York Power Authority 

RCC Chair, January, 2002-December, 2003 

William Longhi Consolidated Edison Company of New York, Inc. 

RCC Chair, January, 2004-December, 2005 

Paul M. Murphy Independent Electricity Market Operator 

RCC Vice Chair, January, 2004-December, 2005 
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Barbara Robertson Ontario Power Generation, Inc. 

RCC Vice Chair, January, 2002-November, 2003 

Howard Tarler New York Department of Public Service 

Ronald J. Halsey 

Chair, NPCC Compliance Monitoring and Assessment Subcommittee 

Members, NPCC Task Force on Coordination of Operation: 


Donald L. Gates ISO New England Inc. 

Chair, NPCC Task Force on Coordination of Operation 
Pat Doran Independent Electricity Market Operator 

Vice Chair, NPCC Task Force on Coordination of Operation 


Glenn W. Brown 
Michael C. Calimano 
Gratien Charest 
John G. Mosier, Jr. 
Clarence Penwell 
Paul A. Roman 
Michael Schiavone 
Daniel Soulier 
Paul Tremblay 
Roger C. Zaklukiewicz 


New Brunswick Power 
New York ISO 
Hydro-Quebec TransEnergie 
Northeast Power Coordinating Council 
Nova Scotia Power Incorporated 
Northeast Power Coordinating Council 
Niagara Mohawk Power Corporation 
Hydro-Quebec TransEnergie 
Hydro One Networks, Inc. 

Northeast Utilities 


Stephen L. Corey New York ISO 

Chair, NPCC Task Force on System Studies 
Larry Eng Niagara Mohawk Power Corporation 

Chair, NPCC Task Force on Coordination of Planning 
Philip A. Fedora Northeast Power Coordinating Council 

Francis J. Flynn, Jr. National Grid USA 

Chair, NPCC Task Force on Infrastructure Security and Technology 
James W. Ingleson New York ISO 

Chair, NPCC Task Force on System Protection 


The NPCC 2003 Blackout Investigative Team initiated its efforts on August 18, 2003, 
by directing creating an ad hoc working group, the NPCC Sequence of Events Working 
Group, to develop the critical sequence of events within the NPCC Areas on the 14 th of 
August. The Working Group was composed of the following individuals: 


Members, NPCC Sequence of Events Ad Hoc Working Group: 


Peter Brandien 
Glenn W. Brown 
Stephen L. Corey 
Thomas J. Dutkiewicz 
Larry Eng 


ISO New England Inc. 

New Brunswick Power 
New York ISO 
ISO New England Inc. 

Niagara Mohawk Power Corporation 
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Michael Falvo 

Independent Electricity Market Operator 

James W. Ingleson 

New York ISO 

Richard Mailhot 

HydroQuebec TransEnergie 

Barry Milton 

New Brunswick Power 

Sasa Mizdrak 

Independent Electricity Market Operator 

John G. Mosier, Jr. 

Northeast Power Coordinating Council 

John Norden 

ISO New England Inc. 

Len Panzica 

New York Power Authority 

Charles P. Rusowicz 

Consolidated Edison Company of New York, Inc. 

Robert Sauriol 

HydroQuebec TransEnergie 

Kateri Sparks 

New York Power Authority 

Howard A. Tarler 

New York Department of Public Service 

Karl Tammar 

New York ISO 

Robert W. Waldele 

New York ISO 


In parallel with the formation of the NPCC Sequence of Events Working Group, the 
BIT directed the NPCC Inter-Control Area Restoration Coordination Working Group to 
assess the restoration efforts of NPCC following the events of August 14 th . The 
members of this Working Group include: 


Members, NPCC Inter-Control Area Restoration Coordination Working Group (CO-11): 


Alden Briggs New Brunswick Power 

Chair, Inter-Control Area Restoration Coordination Working Group (CO-11) 

Gregory A. Campoli New York ISO 

C. Steven Cooper Independent Electricity Market Operator 

Thomas J. Dutkiewicz ISO New England Inc. 

Stephane Henault Hydro-Quebec TransEnergie 

John G. Mosier, Jr. Northeast Power Coordinating Council 


Additionally, the Task Forces on Coordination of Planning and System Studies were 
immediately charged to assist in the simulation of the power system collapse, and all 
NPCC Task Forces were asked to pursue the lessons learned from the blackout. The 
following Working Groups assisted in these assignments: 


Members, NPCC Working Group on Inter-Area Dynamic Analysis (SS-38): 


Philip J. Tatro National Grid USA 

Chair, NPCC Working Group on Inter-Area Dynamic Analysis 


Eric H. Allen 
Navin B. Bhatt 
Stephen D. Boutilier 
Kirit Doshi 
B. George Dunn 
Daniel Fok 
William L. Hann 


New York ISO 
American Electric Power 
Nova Scotia Power, Inc. 

Virginia Dominion Power 
New York Power Authority 
Independent Electricity Market Operator 
PJM Interconnection 
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Donald Izior 
Xiaochuan Luo 
Mohamed Mobarak 
Mahendra Patel 
Roy Pfleiderer 
Si True Phan 
Paul A. Roman 
Lai-Wan Szeto 
Guy V. Zito 


Consumers Energy 
ISO New England Inc. 

New Brunswick Power 
Allegheny Power 

Niagara Mohawk Power Corporation 
Hydro-Quebec TransEnergie 
Northeast Power Coordinating Council 
Hydro One Networks, Inc. 

Northeast Power Coordinating Council 


Members, Major System Disturbance Task Force (MSDTF): 


Eric H. Allen 
Stephen D. Boutilier 
Robert W. Cummings 
Curt Dahl 
Kirit Doshi 
B. George Dunn 
Danny Fok 
Bill Harm 
Don Idzior 
Donal J. Kidney 
John Lawhorn 
Xiaochuan Luo 
Mohamed Mobarak 
Rob O’Keefe 
Mahendra Patel 
Si True Phan 
Paul A. Roman 
Charles P. Rusowicz 
Philip J. Tatro 
Phil Yum 
Jeff Webb 
Mohsen Zamzam 


New York ISO 
Nova Scotia Power, Inc. 

North American Electric Reliability Council 
Keyspan Energy * (representing LIPA) 

Virginia Dominion Power* 

New York Power Authority 
Independent Electricity Market Operator 
PJM Interconnection 
Consumers Energy 

Northeast Power Coordinating Council 
Midwest ISO * 

ISO New England, Inc. 

New Brunswick Power 
American Electric Power 
PJM Interconnection 
TransEnergie 

Northeast Power Coordinating Council 
Consolidated Edison Company of New York, Inc. 
National Grid USA 
Tennessee Valley Authority 
Midwest ISO * 

Consolidated Edison Company of New York, Inc. 


* Participated in base case development 


Members, MAAC-ECAR-NPCC (MEN) Working Group: 


John Andree 
Robert W. Cummings 
Curt Dahl 
Kirit Doshi 
Ugo Felli 
William L. Hann 
Larry Hodge 


International Transmission Company 
North American Electric Reliability Council 
Keyspan Energy (representing LIPA) 
Virginia Dominion Power 
Independent Electricity Market Operator 
PJM Interconnection 
PJM Interconnection 
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Liana Hopkins 

New York Power Authority 

Donal J. Kidney 

Northeast Power Coordinating Council 

Ken Layman 

New York ISO 

Xiaochuan Luo 

ISO New England, Inc. 

Scott Lockwood 

American Electric Power 

Olushola Lutalo 

Grid America 

Mohamed Mobarak 

New Brunswick Power 

Dale Osbom 

Midwest ISO 

Mahendra Patel 

PJM Interconnection 

Si True Phan 

TransEnergie 

Paul Roman 

Northeast Power Coordinating Council 

Charles P. Rusowicz 

Consolidated Edison Company of New York, Inc. 

Kim Sauerwine 

PJM Interconnection 

Joseph Smith 

Midwest ISO 
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2.0 Executive Summary 

All Areas of NPCC were operating within the Criteria, Guides and Procedures of the 
Northeast Power Coordinating Council as well as the Operating Policies of the North 
American Electric Reliability Council when the series of cascading contingencies in 
northern Ohio began on the 14 th of August, 2003, ultimately leading to the widespread 
loss of load in Ontario, New York and New England. The Areas did not contribute to 
the various causes of the blackout itself, and thus the immediate focus of NPCC was on 
the restoration effort and a study of various preventive measures which could be 
employed to prevent or mitigate the impact of the kinds of events to which NPCC was 
subjected on August 14, 2003. Further, the assessment of cause is presented in the 
“U.S.-Canada Power System Outage Task Force Final Report on the August 14, 2003 
Blackout in the United States and Canada: Causes and Recommendations.” For this 
reason, the challenge to NPCC was to focus on the following: 

• Determine the critical sequence of events which occurred within NPCC on 
August 14 th , summarized in Section 3.0; 

• Provide the necessary resources to assist in the simulation of the events and the 
determination of the causes of the widespread blackout; 

• Identify lessons learned from the restoration experience; and 

• Identify measures which can be implemented in the future to prevent or mitigate 
a similar event in the future. 

The restoration effort on the part of the NPCC Areas following the power system 
collapse of August 14, 2003, was generally timely and effective; no further collapse of 
the system was generated as the interconnected bulk power system was re¬ 
synchronized, and virtually all demand was restored within thirty hours. Despite the 
problems encountered during the restoration process, the NPCC system operators 
restored their respective systems in a timely, orderly and professional manner. The 
system operators of NPCC are to be commended for their actions during a difficult and 
demanding restoration. Further, the training programs within the Areas demonstrated 
their effectiveness in dealing with the wide range of challenges faced in restoring the 
NPCC systems. It must also be recognized that the overall restoration effort in NPCC 
was successfully realized in large measure by having in place mandatory reliability 
criteria requiring restoration plans within each NPCC Area as well as the testing of the 
key facilities and associated critical components necessary to establish a basic minimum 
power system for the purposes of restoration following a major system interruption. 
NPCC also significantly benefited on August 14 th due to the clear lines of authority 
present in each NPCC Area directing the control of the bulk power system as well as 
the restoration process itself. A synopsis of the NPCC report on restoration, 
“Restoration of the NPCC Areas Following the Power System Collapse of August 14, 
2003,” is provided in Section 5.0, and the recommendations of the report are presented 
in Section 6.0. 
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NPCC also actively assisted, and continues to do so, in the enormous task of simulating 
and analyzing the events of August 14, 2003. Section 4.0 details the efforts of NPCC in 
these tasks. 

With the efforts concluded and put into place, the NPCC 2003 Blackout Investigative 
Team relinquishes all follow-up activities to the NPCC Reliability Coordinating 
Committee (RCC). The RCC will continually monitor the implementation within 
NPCC of all recommendations stemming from the studies cited in this report, and it has 
already put into place a rigorous tracking system to ensure that the recommendations of 
the U.S.-Canada Power System Outage Task Force Final Report on the August 14, 

2003 Blackout in the United States and Canada: Causes and Recommendations as well 
as those recommendations developed by the North American Electric Reliability 
Council in the aftermath of the blackout of August 14, 2003, are carried out in a timely 
manner within NPCC. 
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3.0 Sequence of Critical Events in NPCC During the Power System Collapse of 
August 14. 2003 

On August 14, 2003, a cascading series of transmission and plant contingencies outside 
the boundaries of NPCC led to major power outages in Ontario, New York and parts of 
New England as well as the separation of the majority of New England and all of the 
Maritimes from the remainder of the system. In addition, although Hydro-Quebec 
TransEnergie (TE) lost a portion of its exports to the IMO, the NYISO and the ISO-NE 
at the time of the disturbance, the TE transmission system sustained the loss of these 
exports without significant impact. 

The Reliability Coordinators of NPCC had no advance warning of the problems 
evolving in the Eastern Interconnection beyond the borders of NPCC. The sequence of 
events experienced by the NPCC Region culminated in the break-up of the bulk power 
grid and loss of 48,000 MW of load in NPCC within the span of a few seconds. 

At approximately 16:09 EDT, the NPCC region experienced a significant change in 
power flows and frequency, which would prove to be the first indication of the events to 
follow. Prior to the appearance of this first swing, power flows in the NPCC Region, 
including its portion of the Lake Erie transmission loop, were typical for the summer 
period, and within acceptable limits. Transmission and generation facilities were in a 
secure state in all of NPCC. The critical events that occurred within the NPCC Areas 
were as follows: 

At about 16:09, an increase in power flow from Pennsylvania, through New 
York and Ontario, and into Michigan was experienced. 

Shortly afterward, between 16:10:38 and 16:10:41, a sudden large surge of 
power from Pennsylvania, through New York and Ontario, and into Michigan, 
concurrent with a sudden extraordinary increase in system frequency to 60.3 Hz, 
resulted in the beginning of circuit trips along the border between the 
Pennsylvania-New Jersey-Maryland Interconnection (PJM) and the NYISO. At 
the same time, power surged into New England and the Maritimes region of 
Canada. The combination of this power surge and the frequency rise resulted in 
the rejection of 380 MW of pre-selected Maritimes generation due to the 
operation of the New Brunswick Power “Loss of Line 3001” special protection 
system monitoring the status of system frequency and export flows on the 345 
kV interconnection between the Maritimes and New England. 

Between 16:10:39 and 16:10:44, the western PJM-NYISO 345 kV and 230 kV 
interconnections rapidly tripped in a west to east direction, followed 
immediately by the trip of the Branchburg - Ramapo 500 kV tie into the 
southeastern New York region. One out of three in-service tie lines between 
Ontario and Michigan was lost when Keith - Watennan 230 kV line tripped. 
Northwestern Ontario also separated from the rest of Ontario at this time, 
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leaving Northwest Ontario loads fed from the Eastern Interconnection via the 
Manitoba and Minnesota systems. Northeast New Jersey separated from the 
remainder of New Jersey, but remained tied to New York through the South 
Mahwah - Waldwick 345 kV and Hudson - Farragut 345 kV circuits. NPCC 
was then separated from the Eastern Interconnection but continued to supply 
power, via Ontario, into the eastern Michigan and northern Ohio load pocket, 
which had already separated from the Eastern Interconnection. 

Over the period 16:10:46 to 16:10:49, a large power surge from New England to 
New York on their tie lines resulted in the effective separation of these Areas. 

At the same time, Vermont lost approximately 140 MW of load, and southwest 
Connecticut became isolated onto the New York system. The remainder of the 
New England Area and the entire Maritimes Area continued to operate as an 
electrical island. Within the same time interval, the State of New York 
effectively separated into eastern and western islands along the Central 
East/Total East transmission interfaces, with northeast New Jersey and 
southwest Connecticut tied to the eastern island. 

Over the time period from 16:10:49 to 16:10:50, frequency declined below 59.3 
Hz in parts of NPCC, initiating, as designed, automatic underfrequency load 
shedding in Ontario, eastern New York and southwestern Connecticut. In 
addition, large power swings between New York and Ontario resulted in the loss 
of nine internal 230 kV lines in Ontario, separating most of Ontario from New 
York and leaving the Ontario Beck and Saunders hydro stations radially 
connected to the western New York electrical island. 

Between 16:10:50 and 16:10:56, the isolation of the Ontario hydro units onto the 
western New York island, coupled with underfrequency load shedding in the 
western New York island, caused the frequency in this island to rise to 63.0 Hz. 
At the same time, the now extremely generation deficient Ontario island 
frequency continued to decline towards 58.8 Hz, the threshold for the second 
stage of Underfrequency Load Shedding. 

Between 16:10:56 and 16:11:11, three 230 kV lines re-closed automatically in 
Ontario, re-connecting western New York to Ontario; a few seconds later, Stage 
II of the underfrequency load shedding program activated in both Areas. After 
fourteen seconds the three lines that had automatically re-closed tripped a 
second time. With Ontario still supplying 2,500 MW to the Michigan-Ohio load 
pocket (until 16:11:57, when two remaining tie lines between Ontario and 
Michigan tripped), the Ontario system frequency declined towards a widespread 
shutdown at 16:11:58, resulting in the widespread loss of power to Ontario, 
including the cities of Toronto and Ottawa. 

At 16:11:22, New England’s Long Mountain to Plumtree 345 kV line tripped, 
leaving Southwest Connecticut connected to Northport, New York (Long 
Island), through the Norwalk Harbor to Northport 138 kV (circuit #1385) 
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submarine cable across Long Island Sound. In the next second, at 16:11:23, the 
two 345 kV circuits between the ConEdison and Long Island Power Authority 
systems tripped. The LIPA southwestern Connecticut areas remained connected 
through the Northport - Norwalk Harbor 138 kV circuit, but separated from the 
rest of the eastern NY island. At 16:11:45, the Northport - Norwalk Harbor 
circuit tripped, leaving the generation deficient southwest Connecticut and Long 
Island areas destined to collapse. 

Within the western New York island, the boundary of which was slightly 
modified as a result of the re-closure of Fraser to Coopers Corners 345 kV at 
16:11:23, the 345 kV system remained intact between the Niagara and Utica 
areas, and the St. Lawrence area remained connected to the Utica area through 
both the 765 kV and 230 kV circuits. Ontario’s Beck and Saunders generation 
remained connected to New York at Niagara and St. Lawrence, respectively, and 
this island stabilized and remained intact throughout the disturbance. 

As a result of the severe frequency and voltage changes, many large units in 
both New York and Ontario tripped. The eastern island of New York lost the 
heavily populated areas of southern New York, including New York City, Long 
Island and the northern New York City suburbs. At 16:11:29, the Edic - New 
Scotland 345 kV circuit reclosed, and the New Scotland to Leeds 345 kV path 
tripped, splitting the northern and southern sections of the eastern New York 
island, and reconnecting the remaining Albany area load to the western New 
York island. 

At this point Ontario, New York and New England began the task of stabilizing the 
remaining electrical islands, restoring the transmission system and ultimately restoring 
customer load. 
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4.0 NPCC Study and Simulation Efforts of the Power System Collapse of 
August 14. 2003 

The system studies portion of the Blackout Investigation consisted of the replication of 
the events that occurred on August 14, 2003 and “what if’ analyses. This investigation 
work included: 1) steady state analysis using powerflow simulation and 2) dynamic 
simulation and analysis. 

The powerflow simulation work was conducted by the MAAC-ECAR-NPCC (MEN) 
Working Group under the auspices of NERC. A large portion of the simulation and 
system analysis work was done by the NPCC representatives. The simulations for this 
first portion of the Blackout Investigation were conducted for conditions from 15:00 to 
16:05 EDT. Following the outage of the Sammis-Star 345 kV circuit at 16:05 EDT it 
was no longer possible to obtain powerflow solutions and dynamic simulation was 
required to replicate system conditions after that time. Critical events modeled include: 

• Conditions prior to the first major transmission outage including the outage of 
the Eastlake 5 Unit in the Cleveland area. 

• Outage of the Harding-Chamberlin 345 kV circuit at 15:05:41 EDT. 

• Outage of the Hanna-Juniper 345 kV circuit at 15:32:03 EDT. 

• Outage of the Star-South Canton 345 kV circuit at 15:41:35 EDT. 

• Outage of various underlying 138 kV circuits between 15:39 and 15:59 EDT. 

• Outage of Sammis-Star 345 kV circuit at 16:05:57 EDT. 

The overall powerflow study work included: 

• Development and benchmarking of base cases for conditions prior to each major 
outage. The benchmarking involved comparisons to actual recorded system 
data. 

• Simulation of the critical contingencies. 

• Analysis of powerflow simulation results and “what if’ testing. This work 
involved the development of comparison tables and graphical portrayal. 

The steady state analysis work was conducted starting from immediately after the 
blackout in mid-August 2003 to December 2003. The results from this study work 
identified that, immediately after the first major outage, the system in the northeast 
Ohio area was above limits on a first contingency basis. They also identified that the 
loss of the Eastlake 5 Plant was critical to reactive power supply and voltage 
performance of the Cleveland area system. Results were included in the US-Canada 
Power System Outage Task Force Interim and Final Reports. In addition, presentations 
and direct input were provided from this work at the Blackout Technical Conferences in 
December 2003 in Philadelphia and January 2004 in Toronto. 

The dynamic simulation work was conducted by the Major System Disturbance Task 
Force (MSDTF) also under the auspices of NERC. This group was essentially the 
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NPCC SS-38 Working Group including the necessary outside participants that normally 
would contribute to SS-38 studies. The dynamic simulation work started at the time 
when the steady state modeling work could no longer continue, which was 16:05:57 
EDT. This was judged to be the point at which the cascade began, eventually leading to 
the large area blackout. The results of this study work were included in presentations 
and were provided as direct input at the Blackout Technical Conferences in December 
2003 in Philadelphia and January 2004 in Toronto and at the New York and Ontario 
Technical Workshops in New York City during April, 2004 and in Toronto during July, 
2004. 

The overall dynamic simulation study work included: 

• Development of the dynamic simulation case corresponding to the time of the 
Sammis-Star outage. 

• Fine-tuning the dynamics model. This involved extensive work to ensure that 
all data used to model generators and control systems was valid and reasonable. 
This included excitation systems, governors, power system stabilizers, HVDC 
systems. Furthermore, this included the data used in standard program models 
and user written models. This work was essential to prevent bad data from 
causing numerical errors that would terminate simulations or give erroneous 
results. 

• Simulation of all events from 16:05:57 EDT through 16:11 EDT, benchmarking 
of the simulation results versus recorded data and analysis of results. Simulation 
through this time period includes the massive power swing through New York 
and Ontario and its immediate reversal, generators losing synchronism and 
slipping poles in the Detroit area and the fonnation of islands beginning at 
16:10:45 EDT. 

• Analysis of the eastern New York island, the western New York island and the 
Ontario island by consideration of load-generation imbalance prior to the 
formation of the islands and in each island as the underfrequency load shedding 
stages operated and as generators were lost. 

The dynamic simulation work started in October 2003 and is still continuing. The 
results of the work showed exceptional correspondence between the simulations and the 
recorded data through time 16:10:42. Around 16:10:42 the loss of synchronism of a 
group of Detroit area units required a great deal of effort by the MSDTF to model and 
properly benchmark. Simulations past that time to 16:11 EDT have been benchmarked 
and mark the beginning of the NPCC TFSS study work. 

The NPCC TFSS Blackout Study includes the following tasks: 

• Compilation of data for the studies. This has already been completed as part of 
the work done to this time. 

• Replication of events through the period of study. As indicated above, this work 
has been completed to time 16:11 EDT, but the investigation of the conditions in 
the islands that had formed will continue past time 16:11. 
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Evaluation of system and equipment performance during island formation, 
causes of islanding, and related possible mitigation measures. 

Evaluation of post-islanding performance and related possible mitigation 
measures. 

Evaluation of potential preventive mitigation measures. 
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5.0 Summary of the NPCC Restoration Efforts 

As each NPCC Control Area was in the process of assessing the current state of its area 
and that of its neighboring Control Areas, efforts immediately began to initiate the 
restoration process. The restoration effort following the power system collapse of 
August 14, 2003, was generally timely and effective. No further collapse of the system 
was generated as the interconnected bulk power system was re-synchronized, and 
virtually all demand was restored within thirty hours. The NPCC Region separated 
from the Eastern Interconnection primarily at the interfaces between PJM and the 
NYISO and between Michigan and the IMO. Internal to NPCC, the IMO was the most 
severely impacted Control Area, with a large portion of the province blacked out; only 
northwestern Ontario and several small load and generation pockets remained in 
service, with generation facilities at each end of Lake Ontario remained connected to 
Ontario and New York load. The NYISO experienced a widespread outage in the 
southeastern portion of New York state, south and east of the Total East interface. The 
NYISO separated from the Eastern Interconnection predominantly along the PJM- 
NYISO interface, with the exception of some northern New Jersey load that was left 
isolated on the NYISO system. Two large hydro facilities and the Hydro-Quebec HVdc 
tie were the major facilities that remained in service, providing power to the remaining 
upstate load and the isolated northern New Jersey load. The ISO-NE separated from the 
NYISO primarily along its interface with New York, isolating the bulk of the ISO-NE 
and the Maritimes Control Area from the Eastern Interconnection. Portions of 
southeastern Connecticut remained connected to New York and collapsed with the 
southeastern New York area, and some New York load remained connected to the 
Vermont system of the ISO-NE. With the exception of the load lost in southeastern 
Connecticut and smaller pockets in Vermont, the ISO-NE remained largely intact. 
Within the Maritimes Control Area, New Brunswick Power observed the operation of 
its “Loss of Line 3001” special protection system due to the frequency excursions 
experienced. 

From an NPCC perspective, the initial focus was to re-synchronize to the Eastern 
Interconnection, with the NYISO attempting to re-establish ties with PJM. The NYISO 
initiated communications with PJM and successfully re-synchronized with the Eastern 
Interconnection at 18:52 on August 14 th through an automatic synchro-check relay on 
the Erie East (PJM) to South Ripley (NY) 230 kV tie line. The second tie, and the 
primary interconnection to PJM and the Eastern Interconnection, was established at 
19:06 through the Ramapo 345 kV station, providing a stronger link between the PJM 
500 kV transmission network and the NY 345 kV transmission network and 
accordingly a stronger system to support the continuing NY and IMO restoration 
process. 

Following this effort, the ISO-NE and the NYISO worked to re-synchronize New 
England to New York. The two Control Areas agreed to make the initial attempt 
through the Alps (NY) to Berkshire (NE) to Northficld (NE) tie, with the 
synchronization at the Northfield generating station. In an effort to gain better voltage 
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control to support the effort, the NYISO energized a 345 kV loop from its southern tier, 
into the Albany area (Alps) and continuing through the northern portion of the Con Ed 
service territory. At 01:53 on August 15 th , the ISO-NE and the NYISO successfully re¬ 
synchronized at the Northfield station. At this point, all of the NPCC Areas, the IMO, 
the NYISO, the ISO-NE, Quebec and the Maritimes Area were now successfully re¬ 
connected with the Eastern Interconnection. As the restoration effort continued in the 
IMO, the NYISO and the ISO-NE, Area to Area tie flows were carefully monitored to 
ensure continuing load and generation as the restoration proceeded, and emergency 
assistance was provided among the NPCC Areas as required and as available. 

For the Independent Electricity Market Operator (IMO-Ontario Area) and the New 
York Independent System Operator (NYISO-New York Area), which were severely 
impacted by the power system collapse, a critical element to their successful restoration 
was the rapid restoration of the basic minimum power system. A basic minimum power 
system is a transmission system that consists of one or more generating stations, 
transmission lines, and substations operating in the form of an island, permitting it to be 
re-started independently and subsequently re-synchronized to other islands or the main 
bulk power system. The transmission elements of the basic minimum power system 
connect units which have blackstart capability to those units without blackstart 
capability and include selected tie lines and corresponding sub-stations judged to be 
essential to the formation of the larger power system. 

The two Areas of NPCC which were severely disrupted on August 14 th established their 
respective basic minimum power systems as follows: 

IMO 13 hours 

NYISO 10 hours 

Ultimately, the transmission systems in all three impacted NPCC Areas were effectively 
restored in their entirety as follows: 

IMO 26 hours 

New York ISO 30 hours 

ISO-NE 9 1/2 hours 

The restoration times above reflect the extent of the restoration efforts required within 
each Area. 

The overall restoration effort in NPCC was successfully realized in large measure by 
having in place mandatory reliability criteria requiring restoration plans within each 
NPCC Area as well as the testing of the key facilities and associated critical 
components necessary to establish a basic minimum power system for the purposes of 
restoration following a major system interruption. NPCC also significantly benefited 
on August 14 th due to the clear lines of authority present in each NPCC Area directing 
the control of the bulk power system as well as the restoration process itself. Rigorous 
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and enforceable criteria, and well defined authority, must be retained as the industry 
transitions to the NERC Reliability Functional Model. 

Nevertheless, the complexity of reliably restoring supply to 48,000 MW of load in a 
timely manner identified the need for improvements in various areas, which are 
captured in the recommendations following in section 6.0. 
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6.0 NPCC Recommendations 


The following recommendations were derived by the NPCC Inter-Control Area 
Restoration Coordination Working Group in consideration of those detennined from the 
individual Area reports and the contributions from system operators attending the 
NPCC System Operators Seminar held on May 5 and 6, 2004. It should be noted that 
there is no implied hierarchy in the order in which they are presented. 

These recommendations are considered to be applicable to all NPCC Areas on a 
Region-wide basis, and they were approved by the NPCC Reliability Coordinating 
Committee on July 14, 2004. 


Procedures and Training 
Synchronizing of Electrical Islands 

Guidelines for matching voltage and frequency for the manual re-synchronization of 
electrical islands to be developed and incorporated in the “NPCC Inter-Control Area 
Power System Restoration Reference Document.” These guidelines would then be 
incorporated in each Area restoration plan to facilitate Area to Area re-synchronization, 
where applicable. 

Inadvertent Re-Synchronization 

Each Area to review the synchronizations made between electrical islands by automatic 
re-closing and determine if these re-closures were: a) appropriate; b) consistent with 
the normal, steady state design intent of the automatic re-closing systems; c) acceptable 
for the rare event which occurred on August 14, 2003. The inadvertent 
synchronizations done manually will be investigated, and methods to avoid manual 
inadvertent synchronizations in the future should be identified. Results are to be 
incorporated into switching procedures and training. 

Stabilization of Surviving Electrical Islands 

Each Area to review its restoration plan to address the actions necessary to stabilize 
operations in the remaining electrical islands following a major system separation. 

Load Shedding 

Each Area to ensure that its load shedding capability remains, where applicable, viable 
in restoration situations following a major system disruption. 

Operator Authority 

Each Area to continue to emphasize in its system operating procedures, job descriptions 
and operator training that its system operators possess the authority to take any action 
required, including load shedding, to comply with the NPCC Criteria and NERC 
requirements. 
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Restoration Training 

NPCC to develop plans for inter-Area restoration training drills, including those 
participants critical to restoration (such as Transmission Operators and Satellite Control 
Centers), simulating the restoration, the scope of which can include whole or partial 
Areas. 


Communications 

Communications Management 

Each Area to review its voice telecommunication facilities and procedures to identify 
means to better; 1) manage call volume infonnation, 2) prioritize communications and, 
3) disseminate necessary information during major system emergencies. 

Tools 

Alarm Management 

Each Area to review the ability of its Energy Management System (EMS) to buffer and 
prioritize alarms during a major system disturbance. 

Wide Area View 

Each Area to provide its system operators with enhanced capabilities to permit a wide 
area view which will permit a more rapid assessment of the state of the interconnected 
bulk power system following a large scale system disturbance. 

Criteria and Compliance 

Testing of Key Facilities and Associated Critical Components 

NPCC Document A-03, “Emergency Operation Criteria,” defines a comprehensive 
program to identify, monitor and test the key facilities and associated critical 
components required to establish a basic minimum power system for purposes of 
restoration. This testing program to be further strengthened by incorporating these 
criteria requirements in NPCC Document A-08, “NPCC Reliability Compliance and 
Enforcement Program.” 

Restoration Criteria and Guides 

NPCC Document A-03, “Emergency Operation Criteria,” and the “NPCC Inter-Control 
Area Power System Restoration Reference Document” to be reviewed to incorporate 
lessons learned from the restoration efforts. 

Fuel Supply for Emergency Generators 

NPCC Document A-03, “Emergency Operation Criteria,” to be modified to add a 
requirement to address adequate on-site fuel supplies for stand-by emergency 
generators associated with key facilities for restoration. 
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Appendix A 

Glossary of Available Reports on the Power System Collapse of August 

14,2003 

Report Internet Address 

“U.S.-Canada Power System 
Outage Task Force Final Report on 

the August 14, 2003 Blackout in https://reports.energy.gov/BlackoutFinal-Web.pdf 
the United States and Canada: 

Causes and Recommendations” 

Restoration of the NPCC Areas 

Following the Power System https://www.npcc.Org/publicFiles/blackout/currentYear/R 

Collapse of August 14, 2003 estoration_of_the_NPCC_Areas.pdf 

“IMO August 2003 Blackout- 

Restoration Evaluation Report” http://www.theimo.com/imoweb/pubs/corp/EPTF_Aug20 

03_RestorationEvalRpt.pdf 

“Report of the NYISO Restoration 

Working Group on the August 14, http://www.nyiso.com/topics/outage2003_info/investigati 
2003 System Event” 0 ndata/nyiso rwg report_ 11 na 1.pd(' 

“Blackout 2003-Performance of 

the New England and Maritimes http://www.iso- 

Power Systems During the August ne.com/special_studies/August_Blackout_Report/ 

14, 2003 Blackout” 
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